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ABSTRACT 

Researchers at Utah State University *s special education 
department compared the performance of student teachers on dimensions that 
reflected pedagogical practices known to promote effective teaching of 
special education students. This study evaluated and compared dimensions of 
special education teaching performance among student teachers from three 
teacher preparation programs: traditional on-campus, distance education, and 
alternative preparation. The university developed a comprehensive performance 
evaluation tool using the Special Education Student Teaching Evaluation Form 
as the foundation for the new instrument. The student teaching evaluation 
consists of 19 items. An observation sheet collects frequency counts, student 
seating arrangement, and duration data. A lesson plan is obtained from the 
student teacher at the beginning of the observation. Five items are observed 
during the new material and guided practice phase of the student teacher’s 
lesson: sufficient opportunities to respond, uses appropriate correction 
procedures, sufficient rate 'of reinforcement, pupil engagement, and uses 
class time efficiently. Pupil engagement and uses class time efficiently are 
also observed during independent practice, as is active monitoring. After the 
observation, student teachers complete an interview on various aspects of the 
lesson and instruction. Researchers plan observe and evaluate all student 
teachers at least once before the end of the semester, conduct reliability 
checks on 20 percent of the observations, and analyze the data over the 
summer. (Contains 10 ref erences . ) (SM) 
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COMPARATIVE EVALUATION OF ALTERNATIVE, DISTANCE, AND TRADITIONAL TEACHER 

PREPARATION STUDENT TEACHERS 

A recent review by McLeskey, Smith, Tyier, and Saunders (20O2) describing reports reieased by the 
United States Department of Education and other agencies confirms the “severe, chronic shortage of 
speciai education teachers in the United States" (p.10). According to the review, ninety-eight percent of 
school districts reported a shortage of special education teachers. In the 200 largest cities in the United 
States special education was the educational area reporting the greatest need for teachers. As a result of 
the shortage, 10 percent of individuals teaching in special education classrooms are not certified to teach 
special education. 

The critical, continuing nature of this shortage has challenged colleges of education to extend their means 
of recruiting and training special education teachers, often to the benefit of rural communities. Traditional 
on-campus programs have been augmented by on-line courses, distance education, and the accelerating 
demand has contributed to the current proliferation of alternative routes to certification. Traditional 
college-based pre-service certification programs are those that have a combination of subject matter and 
pedagogy as coursework, with standards developed by NCATE and CEC, and are professionally guided 
and state approved (Nougaret and Scruggs, 2002; Rosenberg and Sindelar, 2001). ARC programs differ 
in significant ways from traditional programs and among themselves. They vary from internships for 
individuals whose initial degree is outside of special education, to district programs for paraprofessionals 
or substitute teachers. Often institutes of higher education (IHEs) and State Offices of Education and/or 
school districts form partnerships that provide alternative certification programs. In some states 
educational entities may obtain State Office of Education approval without IHE involvement. 

Quality of ARC programs range from state of the art to poor. Results of studies evaluating quality of ARC 
programs have been mixed, with traditionally prepared teachers often outperforming ARC students, yet 
ARC programs often do meet state or district standards (Rosenberg and Sindelar, 2001). Given the 
current and anticipated special education teacher shortage, and the ardent support of ARC by the United 
Sates Office of Education (United States Department of Education, 2002), ARC programs will continue to 
increase. 

The reliance on ARC programs is impressive in many states. For example, Texas currently certifies more 
special education teachers through alternative methods than traditional (Geiger, Crutchfield, and Mainzer, 
2002). The quality teaching has recently received renewed focus (United States Department of 
Education, 2002). When considering how to prepare quality ARC teachers, the question of quality 
becomes; What components of ARC programs are required to ensure quality teachers? This question is 
identical to what is asked of traditional teacher preparation programs, and the answer appears to be 
identical. Pointing out the similarity of program goals across program types. Roach and Cohen (2002) 
suggest the following key components of quality ARC programs: 1) Pedagogical and content knowledge; 
2) Rigorous entrance requirements; 3) Linking K-12 schools with teacher preparation curriculum; 4) 
Clinical experiences; 5) Standards. Given similar quality program components, how do the graduates of 
program types compare? 
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Effective teaching techniques are those specific pedagogical practices that have been demonstrated to 
improve pupil-leaming outcomes (e.g., Brophy & Good, 1986; Hofmeister & Lubke, 1990; Rosenshine & 
Stevens, 1986). A summary of indices of effective teaching is provided by Rosenshine and Stevens 
(1986); 

the major components in systematic teaching include teaching in small steps with student 
practice after each step, guiding students during Initial practice, and providing all students 
with a high level of successful practice... all teachers use some of these behaviors some 
of the time, but the most effective teachers use most of them almost all of the time (p. 
377). 

Hofmeister and Lubke (1990), also using the research from Rosenshine and Stevens (1986), identified 
five phases that comprise the effective teaching cycle (daily reviews and prerequisite checks, 
presentation of new material, guided practice, independent practice, and weekly and monthly reviews). 
They additionally describe a model, test, retest procedure to correct student errors, a procedure further 
developed by Martella, Nelson, and Marchand-Martella (2003). Rosenshine and Stevens (1986) found 
that during guided practice, effective teachers spend more time asking a large number of questions and 
providing multiple repetitions of opportunities to respond correctly. Good, Grouws, and Ebmeier (1983) 
suggested that less effective teachers daily covered 37% less material than more effective teachers. 

Utah State University’s (USU) Special Education Department chose to compare the performance of 
student teachers on dimensions that reflect pedagogical practices known to promote effective teaching of 
special education students. The current study evaluates and compares dimensions of special education 
teaching performance among student teachers from three USU Mild/Moderate teacher preparation 
programs; traditional on-campus, distance education, and alternative preparation. While originating from 
the same department, these three programs have different delivery systems. Ail have rigorous entrance 
requirements. They share performance standards (NCATE and CEC, along with Utah State Office of 
Education approval), and have extensive field experiences and school linkages. Given the adherence to 
national and department standards, pedagogical and content knowledge are similar. Courses in the 
traditional and distance programs are delivered over a two-year period. The traditional on-campus 
program is mirrored in the distance education program as it provides identical courses delivered by 
means of an Internet based teleconferencing technology. This uses two-way compressed audio/video 
with courses taught by USU special education faculty. All students are supervised in their practice and 
student teaching by USU faculty. USU's Alterative Teacher Preparation (ATP) program is designed for 
individuals who have a bachelor’s degree and are willing to teach in a special education classroom during 
the one-year training period prior to certification. Pedagogical and content courses are modified to fit the 
compressed schedule while classroom teaching experiences are much more intense than in the other two 
programs. School district personnel collaborate with USU faculty to shape the course modifications and to 
supervise the students in their teaching assignments. 

USU's development of a comprehensive performance evaluation tool began when a literature review 
uncovered the lack of valid and reliable perfomnance evaluation instruments that could compare student 
teachers across the three program types. USU’s Special Education Student Teaching Evaluation Form 
was chosen as the foundation for a new instrument. Preliminary field testing led to extensive revisions, 
including detailed operational definitions of each item and scoring procedure Teacher performance 
examples used to clarify items were developed from the field trials and supportive field-based materials 
accompanying the original USU form (Lignugaris/Kraft, 1995). During this process two developers 
attended Utah’s Jordan School District training in JPAS (Jordan School District Performance Appraisal 
Systems, version 3, 1996) to refine observational skills and methodology. Collaborative revisions and field 
tests alternated over many months, with several USU faculty contributing to this process. Field testers 
observed student teachers, resolved rating differences, and revised the instrument and evaluation 
process to enhance standardization and increase inter-rater reliability. The creation of procedures to 
structure quantitative data collection during lessons greatly improved the standardization of the 
observations. Refinements to the forms and procedure continued until the developers succeeded in 
achieving 100% agreement in their observation scores. Additional observers will be trained using video 
and classroom observations to 80% agreement. 
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The student teaching evaluation consists of 19 items. Each item is scored separately with minimal 
overlap. An observation sheet is designed to collect frequency counts, student seating arrangement, and 
duration data. A score sheet is used to summarize data by assigning numerical values to each item at the 
conclusion of the observation. The rating scales also include operational definitions and examples to 
guide the scoring. 

A lesson plan is obtained from the student teacher at the beginning of the observation. The quality of the 
lesson plan is judged on whether the entire teaching cycle is structured into the plan, along with sufficient 
teaching and practice, and clear explanations and format. The lesson activities are further judged on 
whether or not they promote lesson objectives. 

Five items are observed during the new material and guided practice phase of the student teacher's 
lesson {sufficient opportunities to respond, uses appropriate correction procedures, sufficient rate of 
reinforcement, pupil engagement, and uses class time efficiently). Pupil engagement and uses class time 
efficiently are also observed during independent practice, as is active monitoring during independent 
practice. These items are recorded on the observation sheet. 

Sufficient opportunities to respond, uses appropriate correction procedures and sufficient rate of 
reinforcement are collected during three minute intervals during the new material and guided practice 
phase of the student teacher's lesson. An opportunity to respond is defined as a teacher initiated 
opportunity for student response. Student responses are recorded as correct or incorrect. Following an 
incorrect response, the student teacher's test and retest behaviors are also recorded. The number of 
opportunities to respond is converted into opportunities per minute and scored accordingly. 

Uses appropriate correction procedures is measured by recording whether or not the student teacher 
uses the entire correction procedure (i.e., model, test, retest), a partial procedure, or no correction. The 
data are converted to a percentage of full or partial error corrections and scored. Sufficient rate of 
reinforcement is evaluated by recording each praise statement made during the three-minute intervals. 
These data are converted to reinforcement per minute rate. 

Pupil engagement is collected between the 3-minute frequency count intervals in which the above data 
are collected. As time permits, three samples are collected during the new material and guided practice, 
and three more collected during the independent practice. Prior to the observation, the classroom seating 
arrangement is diagramed and each student is assigned a number. The students are observed in the 
order of their numbers using a 10 second momentary time sample observation method. Pupil 
engagement data are aggregated to get an average percent of students engaged across all observations. 

Uses class time efficiently is based on data recorded throughout the lesson. Total class time and class 
time not being used to promote academic learning are recorded. The percent of class time wasted is 
subtracted from 1 00% of total time. 

Active monitoring during independent practice records every monitoring interaction the teacher has with 
each student during this phase of the lesson. The classroom-seating diagram is used for marking the 
pupil-specific frequency tallies. After independent practice concludes, the number of student contacts per 
minute and the percent of total students contacted are computed and converted to a numerical score. 

The remaining items are scored upon completion of the observation. Uses effective teaching cycle 
measures how many phases of the effective teaching cycle are demonstrated in the lesson. Clear 
formats, activities, directions, and Interactions with pupils] and positive and enthusiastic are scored 
according to operational definitions. Quality of reinforcement (i.e., contingent, specific to behavior, varied 
across behavior, non-disruptive, sincere, and appropriately distributed), and uses low-key behavior 
management tactics (e.g., planned ignoring, proximity control, eye contact, etc.) are also based on 
operational definitions. Accurate presentation of subject matter examines the accuracy of the factual 
instruction that the student teacher delivers. For example, errors modeled in a math lesson would reduce 
the student teacher’s score. 

After the observation, an interview is conducted between the observer and student teacher to obtain 
further information. Qualitative information is collected to assess some aspects of the lesson and overall 
instruction. First assessed is the extent to which the student teacher follows the behavior management 



ERIC 



176 



plan (e.g., does the behavior management plan enhance learning and include the rules and sequence of 
consequences). ). Uses moment-to-moment performance to influence teaching is observed during the 
lesson and followed up with interview questions. Collecting data relative to lEP objectives, and uses day- 
to-day pupil performance to influence teaching examine how the student teacher collects data and uses it 
to guide curricular decisions. Observations are followed up with evidence supplied during the Interview. 
Data used to modify management plan for class or individuals also considers observational data and 
evidence supplied by the student teacher to demonstrate data collection and its use. 

Between the authoring of this paper and its presentation the observational data collection will have begun. 

The training of observers will consist of: 1) familiarization with the observation form and the protocol for 
it, including a model script for the interview portion of the observation; 2) familiarization with the score 
sheet and the operational definitions associated with it; and; 3) practice in use of the forms using both 
videotaped teaching vignettes and classroom observations. Training will continue with each observer until 
at least 80% point-by-point accuracy is obtained. 

Currently the traditional and the distance education programs each have sixteen student teachers, and the ATP 
program has 53 certification students teaching in mild/moderate classrooms. It is our goal to have all observed and 
evaluated at least once before the end of the semester. The researchers will conduct reliability checks on 20% of the 
observations. Data will then be conpiled and analyzed over the summer. 

In this study, three kinds of special education certification programs with many equivalent key con^onents are 
being compared. This comparison minimizes program quality differences while assessing outcomes according to 
program type. USU and its school district and State Office of Education partners will use the results to improve 
teacher training. Results can also be used to enhance a data-driven vision of possibilities in teacher training. As an 
extension of this study, plans are underway to follow this cohort of newly certified teachers into their first year on 
the job. They will again be observed and evaluated with this instrument. The follow-up will also obtain standardized 
measures of pupil outcomes to ascertain the validity of the teacher measures as indicators of quality, as pupil 
learning is the real end product of teacher preparation and certification procedures. Finally results will be analyzed 
according to location of the teacher to see if there are differential outcomes among USU’s program graduates in 
urban, suburban, and rural settings. 
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